DLBench: a comprehensive experimental evaluation of deep learning frameworks

Abstract Deep Learning (DL) has achieved remarkable progress over the last decade on various tasks such as image recognition, speech and natural language processing. In general, three main crucial aspects fueled this progress: increasing availability of large amount digitized data, affordable parallel powerful computing resources (e.g., GPU) growing number open source deep learning frameworks that facilitate ease development process architectures. practice, popularity calls for benchmarking studies can effectively evaluate understand performance characteristics these systems. paper, we conduct an extensive experimental evaluation analysis six popular frameworks, namely, TensorFlow , MXNet PyTorch Theano Chainer Keras using types DL architectures Convolutional Neural Networks (CNN), Faster Region-based (Faster R-CNN), Long Short Term Memory (LSTM). Our considers different its comparison including accuracy, training time, convergence resource consumption patterns. experiments have been conducted both CPU GPU environments datasets. We report analyze studied frameworks. addition, a set insights important lessons learned from conducting our experiments.